Reply to TR Fenton

I agreewith Fenton and Fenton that the current evidence on the paleo diet might be considered early or preliminary and that further research is critically needed. There is a risk of great societal harm if dietary guidelines and recommendations are issued without a strong foundation of evidence from methodologically robust randomized controlled trials (RCTs), ideally trials comparing different dietary patterns. The systematic review includes only 4 trials with a total of 159 patients (1). This is its major limitation. As Fenton and Fenton point out, this leads to wide CIs, with a lower limit of only 0.04 cm for the waist circumference outcome. This is a clinically unimportant difference. Would anyone even notice such a small loss of fat at their waist? Furthermore, would this outcome correlate with benefits on outcomes important to patients, such as heart disease and stroke? DynaMed Plus downgraded the level of evidence for this systematic review because of surrogate outcomes and also because of the wide CIs that included clinically unimportant differences (2, 3). Whether to focus specifically on which CIs cross the line of no effect (P 0.05) is debatable. Bland and Altman (4) argued recently in the Journal that CIs are “so much more informative [than P values]....If the difference is not significant, the CI gives us a range of possible differences in the population that would be consistent with the data observed”. In a seminal 1986 article comparing CIs with P values, Gardner and Altman (5) concluded with, “Confidence intervals, if appropriate to the type of study, should be used for major findings in both the main text of a paper and its abstract.” Currently available trials of the paleo diet measure mostly surrogate outcomes, at the short term. To generate reliable data on clinically important outcomes, long-term trials are needed. Such trials are expensive but worthwhile, given that the health effects of the “paleo” diet, arguably the world’s most popular diet, is a question of great interest to the public. As Chalmers et al. (6) and others (7) have noted, public funders of research need to be accountable to the needs and interests of the public in a democratic society. If additional RCTs test whether milk products or the elimination thereof contribute to the health benefits of the paleolithic diet, it is imperative they are not funded by the dairy industry. Industry funding of studies of milk and other beverages has been shown to strongly bias conclusions in favor of sponsors’ products (8). Fenton and Fenton suggest that the current trials do not have implications for future research, arguing that the control diets usedwere lowsaturated-fat diets recommended 10–15 y ago. However, the control diets, in fact, reflect current recommendations in dietary guidelines internationally. These dietary recommendations continue to remain largely the same, despite a changing evidence landscape. For example, the next set of US Dietary Guidelines for Americans maintains the 10% recommended cap on saturated fat, despite the fact that relevant evidence does not confirm an association between saturated fats and heart disease (9). These US dietary guidelines are typically adopted internationally. We did not report separate tests against baseline within groups as Fenton and Fenton suggest but rather a comparison of the differences in change from baseline between groups. Hooper et al. (10) also analyzed differences in change from baseline between groups for serum lipid and glucose concentrations in their Cochrane review on saturated fat and heart disease, cited by Fenton and Fenton. Dietary guidelines have not always been based on evidence fromRCTs (11). For example, low-fat dietary recommendations were introduced in 1979 in the United States and in 1983 in the United Kingdom, but a recent systematic review found that there was an absence of supportive evidence from RCTs before their introduction (12). Given that dietary guidelines affect entire populations and that diet-related chronic diseases represent the largest cause of morbidity and mortality in Western countries, government dietary recommendations that are introduced in the absence of supportive evidence from RCTs have the potential to do great harm (10). Investing in and acting on evidence from RCTs comparing dietary patterns is an imperative for the protection of public health.